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FOREWORD 

Our staff at the Education Trust - which includes many former public 
schoolteachersand administrators- spendsa lot of time working with 
educators in schools and classrooms a II over the country. Because 
ourfocusasan organization is on closing the achievement gap that 
separates low-income students and students of color from other young 
Americans, much of the work of our "Practice Team" takesplace in 
schools serving concentrations of such students. In the course of that 
experience we see a lot of prac tic es that seem to represent the very 
best of American education - powerful teaching that results in powerful 
student learning. Butwe also see a lot of practicesthat concern us 
deeply. 

Inside the Education Trust, though, we have anotherteam - our "Data 
Team" - that hasa rather different window into the nation'sschools- the 
window provided by close analysisof student and school-performance 
data. Every year, ouranalystscollect and analyze data from almost 
every public school in America, looking for patterns and identifying 
schools that breakthose patterns. We've been especially focused on 
schools that serve concentrations of poorchildren and children of color 
and produce unusually high student performance. 

Last year, we had a first chance to work across those two teams: to 
start with data on school performance, then look systematically at 
the practices in a set of high schools characterized by unusually high 
performance on the ACT. Conducted jointly with the staff at ACT this 
study- On Course forSuccess- taught usa lot about instruction in 
schoolsatthe highest ends of student performance. But it didn't allow 
usto even begin to answerthe question we find moston the mindsof 
conscientious high school educators tod ay: "Whatshall we do to help 
the kidswho arrive furthest behind?" 

So we are enormously grateful to the Bill & Melinda Gates Foundation for 
giving usthis opportunity to lookclosely, overthe course of nearly a full 
academic year, atthe practices in fourpublic high schools that do an 
unusually good job of "growing" the performance of students who enter 
behind, and compare these to three demographic ally similar schools 
that get more average results. 
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INTRODUCTION 

Ihe National Gove mors Association earlierthisyearsponsored a National 
Education Summit on High Schools that highlighted the urgent need to 
improve American high schools. U.S. Secretary of Education Margaret 
Spellings la id out the sad facts: 

"Improving the quality of high school education isan urgent challenge 
thatcan only be solved by working together, in a bipartisan fashion. 
Getting every child to graduate high school with a meaningful diploma 
in their hands is one of the biggest challengesourcountry faces. 

Today only 68 out of 100 entering ninth-graderswill graduate from high 
school on schedule. Fewerthan 20 will graduate on time from college. 
Meanwhile, 80 percent of the fastest-growing jobswill require some 
postsecondary education." 

While policymakers continue to exert pressure and pour resources into K- 
8 improvement, few realize that better-prepared primary school students 
don't necessarily translate into more - ormore successful - high school 
graduates. In fact, available evidence suggests that even as better- 
prepared students a re moving into high schools, academic growth in our 
high schools isdeclining. 

We know whatthe problem is, but solutions a re harderto come 
by. Information about high school achievement and closing the 
achievementgapsbetween demographic groupsisan emerging field 
in which we have very limited information, with more questions than 
answers. 

This study is meant notto answerall possible questions, but to help 
answerone ofthe most frequent questions we getfrom high school 
staffs: What do we know about the characteristic sand practices 
of schools that are especially effective at improving the academic 
performance of previously low-performing students? 

Ihe high-impactschoolsthat we studied are by no meansgood at all 
things. For example, they are not yet among the highest-performing 
schools in their respective statesoreven where their staffs hope they will 
soon be in terms of student proficiency. In fact, most still have a long way 
to go. But they have been able to make greater-than-expected gains 
with previously underperforming students, something that still eludes most 
American high schools. Because of this foe us, we believe that this study 
will be of particular interest to schools that receive large numbersof 
below-grade level studentsand wantto accelerate their learning. 

Ihe research design used in this study wasdeveloped in the belief 
that schools that are making greater-than-expected gains will have 
characteristic sand practicesthat differfrom demographic ally similar 
schools that make only expected gains. Sometimesthe differences we 
found were subtle - more so, certainly, than if we had compared high- 
impact sc hoolswith low- (ratherthan average-) impact schools. But we 
did find importantdifferencesin the orientation and practices of these 
two sets of schools, and have done our best to describe them in ways 
that will be helpful to those on the front-lines of high school reform. 
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Study overview 

Thisstudy examined seven public high schools. Fourwere "high-impact" 
- that is, they produced unusually large growth among students who 
entered significantly behind. We compared these high-impactschools 
with three average-impact schools with similardemographics. By looking 
at both sets of schools, we hoped to find out whatthe high-impact 
schoolsdo differently than the average-impact schools. 

In orderto find schools that were high-impact, we developed a set of 
criteria that a school needed to meet to be considered 1 : 

• It had to have greater-than-expected growth overthree years; 

• It had to have at least average performance on the state 
assessments in reading and/ormath (While these schoolsdid 
not have to be high-performing, they did have to be within the 
achievement average of the state); 

• Its achievement gapshad to be smallerthan the state average; 

• It had to have a Promoting Powerlndex 2 at orabove the state 
average; and 

• It had to serve 60 percent or more low-income students. If that 
criterion was not met, the school enrollment had to be 50 percent 
ormore non-White and at least 20 percent- but less than 60 
percent - low-income. 

Once these schoolswere selected, we chose three demographically 
similarschoolsthat produce average growth. 

Overthe course of a year, we collected a good deal of data and 
material from both sets of schools, including schedules, student 
transcripts, assignments and the like. We subsequently surveyed 
administrators, teachersand students. Membersof our"practice team" 
also conducted multi-day site visits, including extensive classroom 
observations, aswell asstudent and teacherfoc us groups. 

Upon analyzing the data, we found that high-impact schools have many 
characteristic sin common with average-impact schools. But they also 
differin significant ways. Each practice described may not be evident at 
every high-impact school or every average-impact school. But we found 
that high-impactschools shared a common range of practices, asdid 
average-impact schools. 

The schools that were identified ashigh-impactschoolsare J ackBritt 
High School, Fayetteville, NC; Los Altos High School, Hacienda Heights, 
CA; East Montgomery High School, Biscoe, NC; and Farmville Central 
High School, Farmville, NC. We hope that otherschools will be able to 
learn from them. The schools identified asaverage-impact, although not 
named, were critical to thisstudy, and the Education Trust is grateful for 
their participation. (For profiles of each school, see Appendix B, School 
Profiles.) 

Following are our majorfindings, organized in five "spheres" that the 
Education Trust research tells us influence school practice. 



1 For a more detailed discussion of the selection criteria and process, please see the Methodology section of this report. 

2 The Promoting Power Index, developed by researchers at Johns Hopkins University, is used to approximate a school’s high school 
graduation rate. For more detail, see the Methodology section of this report. 



4 



Gaining Traction, Gaining Ground 



November2005 



Sphere 1: Culture 

• Though both sets of schools serve many low-income students 
who arrive farbehind other students, high-impact high schools 
are clearly focused on preparing students for life beyond high 
school— specifically, college and careers. By contrast, average- 
impact high schools a re more focused on preparing students for 
graduation. 

• In official policy documents, the clearfocusin high-impact schools 
ison academics. Average-impact schools focuson rules. 

• In high-impact high schools, teachersand administrators express 
consistent views a bout achievement-related school goals. In 
average-impact schools, there are administrators and teachers 
with very high expectations, but much less consistency in the 
school asa whole. 

• In high-impact schools, teachersembrace external standardsand 
assessments; in courseswhere such standardsand assessments 
are unavailable, they create them. In average-impact schools, 
teachers simply tolerate these things. 

Sphere 2: Academic Core 

• High-impact schools have consistently higherexpectationsforall 
students, regardlessof students' prioracademic performance; 
and principals, teachers, and counselors take responsibility for 
helping students succeed. 

• In high-impact schools, barriers to high-level course-taking are 
removed. Studentsare encouraged to take on academic 
challenges. In average-impact schools, there are hurdlesto gain 
access to the most challenging courses. 

• Assessment data isused by high-impact schools for future planning, 
such asimproving curriculum ormaking teacher assignments. 
Average-impact schools tend to use data primarily to measure 
past student performance. 

Sphere 3: Support 

• In both high- and average-impact schools, studentswho arrive 
behind get extra instructional time in English and math. But high- 
impact schools provide help in a way that keepsstudentson track 
with college-preparatory requirements. Average-impact schools 
provide the extra help in a way that delaysentry into grade-level 
courses, making it harderforstudentsto complete college-prep 
requirements. 

• In high-impact schools, administrators and teacherstake 
responsibility forensuring that struggling students get the 
additional help thatthey need. At high-impact schools, little is left 
to chance. Average-impact sc hoolsgenerally offerextra help to 
students, but make it optional. 

• High-impact schools have in place early warning systemsto 
identify students who need help before it’stoo late. Average- 
impact schools a re more likely to provide remedial help after 
students have faltered. 

• Counselors in all schools a re involved in scheduling, but counselors 
in high-impactschoolsare considered membersof the academic 
teamsand are responsible for actively monitoring student 
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performance and forarranging help when needed. Counselors 
in average-impact sc hoolsare more likely to get involved with 
students thro ugh referrals. 

• High-impact and average-impact schools both have partnerships 
with businessesand colleges, but high-impact schools use those 
partnerships to aid in student preparation for postsecondary 
opportunities, while average-impact schools tend to use their 
partnershipsfordropout and drug-abuse prevention. 

Sphere 4: Teachers 

• High-impact schools use more criteria than teacher preference 
to make teaching assignments, looking at factors such as past 
student performance and the teachehsarea of study. Teacher 
assignments a re made to meetthe needsof the students, rather 
than the desiresof the teachers. In average-impact schools, 
teaching assignments a re more likely to be determined bystaff 
seniority and teacher preference. 

• School-sponsored support fornew teachersin high-impact schools 
is focused on instruction and curriculum. Average-impact schools 
provide support fornew teachers, but it is more personal and 
social in nature. 

• Administrators at high-impact high schools adjust class sizes to 
provide more attention for struggling studentsand are not averse 
to larger student-teacher ratios for students who are able to work 
more independently. Classsizesin average-impact sc hoolsare 
relatively uniform. 

• Principalsat high-impact high schoolsexert more control overwho 
jo ins their staff than those at average-impact schools. 

Sphere 5: lime and Other Resources 

• High-impact sc hoolsare more deliberate about the use of 
instructional time, arranging available time to help "catch up" 
students who arrive behind. 

• Students who enterninth grade behind in high-impact schools 
spend more time in courseswith substantial reading and/or 
reading instruction than do their counterparts in average-impact 
schools. 

• Overall, the amount of time that studentsspend in "academic" 
classes is a bout the same in both high- and average-impact 
schools. But in high-impact schools, a largerfraction of thattime 
is spent in grade-level or"college-prep" courses, while students 
in average-impact schools spend more time in "support" or 

"remedial" courses. 

• All of the schools in the study say they protect academic time, but 
high-impact schools have more strategiesto efficiently use time 
and are stricter a bout enforcement. 

• Forstudents in both high- and average- impact schools, the senior 
yearhasfeweracademic challengesthan any otheryear. The 
only exceptions to this finding are students in high-impact schools 
who enterninth grade behind and proficient students in average- 
impact schools. These studentsspend a slightly higher percentage 
of time their senioryearin academic coursesthan they do in their 
freshman year, but less than in their sop ho mo re and junior years. 
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One final note. J ust asthis report does not describe the characteristics 
of low-performing schools, it a Iso does not address the characteristics 
of high-peifoiming schools - that is, schools at which students from 
all demographic groupsare testing at proficient oradvanced levels; 
where graduation ratesare high forall students; and all students 
are challenged by a rigorous curriculum that preparesthem for 
postsecondary options. Rather, the findings a bout these high-impact 
schools point to the necessary- but not nearly sufficient - steps schools 
can take toward higherperformance. 
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Methodology 

In orderto study high-impact high schools, the Education Trust first had to 
define criteria for"high impact." Essentially, we chose and applied four 
consecutive filters: 

• We started with schools that had significant populations of 
students of co lor (African American, Latino, and Native American) 
and/oreconomically disadvantaged students. 

• We then identified which of those schools had proficiency levels 
on state math and/orreading assessmentsgreaterthan state 
average levels of proficiency. 

• Once those criteria were met, we looked for schools that had 
achieved greater-than-expected academic growth with 
previously low-performing students, using the state -specific growth 
models described below. 

• Schools that met all three of these criteria then had to also 
demonstrate at least average ability to keep students in school, as 
well asaverage, smallerthan average ordeclining gapsbetween 
their minority/economic ally disadvantaged groupsand their 
White/non-economically disadvantaged groups. 

The final step wasto select comparison schools that matched the high- 
impactschoolsdemographically but had made only average gains. 
Because they had the available data, we chose high schools from North 
Carolina and California 3 . 

Demographic Criteria 

Schools first had to have a significant population of students of color 
and/oreconomically disadvantaged students. The "non-White" group is 
a composite of African-American, Latino, and Native -A meric an students; 
economically disadvantaged studentsare those in the school receiving 
free and reduced-price meals. Fora school to meet ourdemographic 
criteria, itsenrollment had to be at least 60 percent economically 
disadvantaged students. If that was not the case, minorities had to 
account forat least half of the enrollment, and at least 20 percent, but 
less than 60 percent, of students had to come from low-income families. 

Ac hievemerrt C riteria 

Schoolswith significant non-White/economically disadva ntaged 
enrollments a Iso had to exhibit at least average performance on their 
state math and reading assessments. Using 2003 assessment data in 
math and reading, each school had to have an average overall score 
in one subjectthat wasat orabove the state average and an overall 
score nearthe state average forthe othersubject. The schoolsalso had 
to meetthe same criteria foreach of the subgroups(African American, 
Latino, Native American, and economically disadvantaged). 

North Carolina schools were measured on the state's lOth-g rad e math 
and reading exams in 2003. In California, the high school math exit exam 
wasutilized to determine if a school’s students were performing ator 



3 We also had originally selected schools from Oklahoma to participate in the study. However, after several attempts to secure a 
commitment from a limited pool of average-impact schools that met the necessary performance criteria, we were forced to exclude 
Oklahoma from the qualitative analysis. The first wave of quantitative data we collected from the high-impact Oklahoma school, 
however, was used in the preliminary analysis to identify trends in high-impact schools. 
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above the state average in math, although we did not have data for 
each of the subgroupsfor2003. We did, however, have the 2003 11th- 
grade reading overall and subgroup data which were used to determine 
if schoolsand their subgroupsscored at orabove their respective state 
averages. 

Growth Criteria 

The third step in selecting high-impactschoolswasto determine which 
of the schools that had metthe above criteria were making greater- 
than-expected gains. We used growth statistic sc re a ted by each of the 
two statesto determine if a school washigh impact, average impact, or 
ne ithe r. 



North Carolina 



In North Carolina, the growth criteria were based on state-developed 
growth calculationsthat are heavily weighted with reading and math 
scores. UnderNorth Carolina'sABC accountability system, each high 
school isdetermined to have 'Met Expected Growth’ or'Met High 
Growth' if their Weighted Expected Growth Composite isat orabove 
what would be expected based on their students’ priorachievement 
and preparation. 



VJe looked for 
sc hools that were 
making gieaterthan 
expected gains. 



The calculation'sfirst step is to determine how 
the school performed in the current year by 
creating what iscalled an actual average forll 
academic components. The actual averages 
for each high school are calculated using 
the state'sEnd Of Course Exam (EOC) results 
(Algebra I, Algebra II, Biology, Chemistry, English 
I, Geometry, Physical Science, and Physics), Co liege/ University Prep/ 
College Tech Prep (CUP/CTP), the lOth-grade Competency Exam, and 
the school'sdropout rate forthe current school year. The next step is to 
adjust the actual averagesto create an expected average based on 
prior student achievement on state assessments (eighth-grade reading 
and math exams, eighth-grade competency exam, and prerequisite 
EOC Exams) and the percentage of students the school is preparing for 
postsecondary education (Percent CUP/CTP), while taking into account 
the percent of students drop ping out of the school. The differences 
between the students' actual performances and theirexpected 
performances in each ofthe 11 components are then standardized 
and weighted to more accurately reflecteach component's influence 
on a school'sgrowth. The 11 weighted standard expected-growth 
statisticsare then summed to create the weighted expected-growth 
composite. After completing the calculations forboth these stepsit 
can be determined if a school that met each ofthe criteria above had 
'Met Expected Growth.' Sc hools that 'Met High Growth' have expected 
performance on each ofthe eight EOC Exams approximately 3 percent 
higherthan those that are designated 'Met Expected Growth.' If 
the weighted standard expected-growth composite isat orgreater 
than zero the school has'Met Expected Growth' or'Met High Growth' 
depending on which calculation isbeing used. Forthis study each of 
the high- impactschools'Met High Growth,' while the average-impact 
sc hools only 'Met Expected Growth.' 
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California 

ForCalifomia, growth wasdetermined using the state's Academic 
Performance Index (API). It is a measure of how well a school is moving 
toward and beyond the statewide target score of 800. The API measures 
the growth a school makesfrom one yearto the next by setting target 
growth scores which are 5 percent higherthan the previous yeahsAPI. 
The API iscalculated using assessments in the state's Standardized Testing 
and Reporting (STAR) program a swell as the California High School 
Exit Examination (CAHSEE). These assessments include: ELA Standards 
Test, MathematicsStandardsTest, History/Socia I Science Standards Test, 
CAHSEE, the California Achievement Test, Sixth Edition Survey (CAT/ 6), 
and the California Science Standards Test. Schools that made greater 
then average growth in the API from 1999-2003 were considered high- 
impact schools. Schools that made atorjust below average API growth 
were considered average-impact schools. 

Romoting Power Criteria 

Asa checkto make sure that schools were 
not making gains by pushing students 
out (or simply failing to hold on to them), 
schools that metthe demographic, 
achievement and growth criteria also had 
to have a Promoting Power Index (PPI) at 
orabove their state averages. The PPI is used to approximate a school’s 
graduation rate. The promoting power indicator is a straightforward ratio 
of the numberof 12th-graders enrolled in a schoolto the numberof ninth- 
grad ers (or lOth-g rad ers if the school hasa 10-12 grade span, orif an 
unusually small ninth-grade enrollment indicated the recent addition of a 
ninth grade to the school). 

Gap Criteria 

We also required schools to have small achievementgapsasanother 
checkto ensure schoolsare teaching all students. Any school with at 
least 25 percent White enrollment and at least 25 percent enrollment of 
African-American, Latino, and Native -A meric an students had to show 
an achievement gap between one of theirthree non-White groupsand 
the White group'sscoreson math and reading assessments that wasat 
orbelow the state average. If their gaps we re not below average, then 
they had to show evidence of being on trackto close achievement 
gaps using available data. 

Comparison Schools 

The high-impactschoolswere matched with average-impact schools for 
comparison purposes. The average-impact match schools had to be in 
the same demographic quartilesasthe high-impact schools, while only 
meeting expected oraverage growth using the corresponding growth 
measure. Once those criteria were met, the average-impact schools 
had to fall one standard deviation below their matched high-impact 
school in proficiency on the state assessment in reading and math, as 
well asbeing nearthe state average forpromoting power. 

Data Considerations 



We checked to make 
sure that sc hools were 
not making gains by 
pushing students out 
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Ihe high-impact high school selection criteria are by no means 
problem-free. In addition to the small pool of stateswith growth data, 
we encountered issuessuch asa lackof reported data to ensure 
performance levelswere accurately represented. Moreover, because 
true individual student growth can only be determined using student- 
level data, which we do not have, we have inferred student growth 
using cohort data. Although this is a methodologically sound inference, it 
still is not without issue. 

In addition, there are factorswe were unable to controlforin the design 
of the study. For example, all matched sets of schools a re geographically 
in the same state. However, we were not able to identify matched 
schools in the same districtthat metthe required set of performance 
criteria. Therefore, differencesin the characteristics of the school district 
may also influence differences in academic growth. Also keep in 
mind that just because a school was identified as high impact didn’t 
mean that vast numbensof students met state proficiency standards. In 
fact, using ourcriteria, average-impact sc hoolscould have a higher 
proportion of proficient students than high-impact schools. Although 
possible, that wasgenerally notthe case. We found that most schools 
that had greater-than-expected growth also had a higherpercentage 
of stud e nts meeting p rofic ie nc y sta nd a rd s. 



Data Collection 



In the first wave of data collection, we requested thateach school 
send usa set of artifacts that included student-enrollment figures, staff 
information, official school schedule and school calendar, student 
handbook, masterschedule, list of external partners, student transcripts, 
a professional development plan, and a three-week lesson plan 
with syllabi and sample assignments. We also administered surveys 
to administrators(principals, assistant principals, and department 
chairs), teachers, and counselors. We collected information on teacher 
experience and background, counseling programs, administrator 
practices, instructional practices, course information, texts, materials 
and other resources, and school beliefs and practices regarding student 
performance and academic -support programs. 

Ourcourse information data collection focused *|Up mrnnH 1A /aup fif 
only on ninth- and llth-grade teachersand math nws wave Ul 

and English courses. 



data collection was 



done during site 
to the schools. 



The second wave of data collection wasdone 
during site visits to the schools. While atthe 
schools, we conducted follow-up interviews with 
administrators(principals, assistant principals, 
and department chairs), teachers, and counselors. We also performed 
both school and classroom observations, and conducted student and 
teacherfocusgroups. Classroom observations foe used on ninth- and 
11th -grade math and English classes, and we requested that only 
previously low-performing llth-graders participate in the student focus 
group. All survey and interview protocolsare provided in the attached 
appendices. 
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